Normally, when image capture is performed with an imaging apparatus such as a digital camera, limitations to the dynamic range of image sensors result in occasions arising in which there is white out of background, and faces of people are in-filled in black due to, for example, backlighting.
In order to eliminate such white out and black in-fill, a commonly known method to broaden the dynamic range is to synthesize together images captured with plural different image capture conditions that have different light exposure times. In such a method, for example, a short light exposure time image is captured set with a fast shutter speed, and a long light exposure time image is captured set with a slow shutter speed. Then, after performing brightness matching such that the brightness of the long light exposure time image and the short light exposure time image are equivalent, the long light exposure time image and the short light exposure time image are synthesized together by using the bright region of the short light exposure time image and using the dark region of the long light exposure time image. As a result, an unnatural join occurs in the brightness of the synthesized image when brightness matching has not been accurately performed, leading to a deterioration in quality of the synthesized image.
Moreover, when two images with different image capture times are synthesized together as described above, sometimes positional misalignment arises due to camera shake between the two images, and sometimes subject blur occurs due to the subject moving. If images in which camera shake and subject blur have occurred are employed to perform synthesis, then double images appear of the subject, with a deterioration in the quality of the synthesized image. There is a method proposed to eliminate quality deterioration of a synthesized image caused by such camera shake and subject blur by detecting camera shake and subject blur between two images, and then applying a correction during image synthesis. In such a method, an image is synthesized by performing alignment during image synthesis by translation movement of an image by the detected camera shake amount, or by using appropriate pixel values from out of the short light exposure time image and the long light exposure time image in the subject blur region.
A known detection method for such camera shake and subject blur is implemented based on differences in brightness values between two images. There is a need to accurately perform brightness alignment between the two images when camera shake or subject blur is detected based on the brightness value differences. When brightness alignment is not accurately performed, sometimes a large difference in brightness values also arises in regions that have not moved, and sometimes camera shake and subject blur are falsely detected, and inappropriate correction processing performed during image synthesis results in a deterioration in quality in the synthesized image.
As described above, there is a need to perform accurate brightness alignment during synthesizing together the short light exposure time image and the long light exposure time image. The light exposure amount ratio between the light exposure amount of the short light exposure time image and the light exposure amount of the long light exposure time image, which is needed in order to perform this brightness alignment, may be derived from image capture conditions such as the shutter speed. However, there is a possibility of the light exposure amount ratio in the actual captured image being different to the light exposure amount ratio in the derived image capture conditions due to changes in the lighting source environment or errors in the setting values.
There is a proposal for deriving the light exposure amount ratio from actual captured images. For example, there is a proposal for a method in which the light exposure amount ratio between the long light exposure time image signal and the short light exposure time image signal is detected for each of blocks that are formed by portioning up the captured image. In this method, based on the detected light exposure amount ratio for each of the blocks, gain adjustment is performed by block on the long light exposure signal and the short light exposure signal to broaden the dynamic range.
There is also a proposal to compare the peak value detected by a detection section against a threshold value, and to control the light exposure time of the short light exposure time. When the comparison result is within a fixed range, the light exposure time is further controlled such that the peak value matches the threshold value. In such a method, in the threshold values for the peak value detected by the detection section, values are set to widen the gradation of a high brightness subject. Moreover, gain control processing and gamma correction setting is performed by an AGC section so as to give a wide gradation of the short light exposure time image from the histogram distribution of the short light exposure time images. Even in synthesis processing between the short light exposure time image and the long light exposure time image, synthesis is performed at a predetermined rate matched to respective patterns out of patterns of histogram distributions.